Analyzing the Impact of Collection Methods and Demographics for Android’s Pattern Unlock

Building upon previous work on measuring the strength of graphical passwords we report on further analysis of statistical differences between Android’s graphical passwords collected in-lab via pen-and-paper surveys and those collected online via self-reporting on users own mobile device. Additionally, we conducted between group studies of demographic groups for online, self-reported patterns and investigated weather locale (urban vs. suburban vs. rural), handedness (left vs. right), or reported gender (male vs. female) impacts the visual features of self-reported patterns. We find that there exist subtle but statistically significant differences between data collected in-lab using pen-and-paper and data collected on-the-device via selfreporting, as well as differences between demographic groups that should be considered when recruiting participants. For example, we found subtle left/right shifting between genders in the Android pattern’s grid space as well as the presence or absence of visual features, such as crosses, within the pattern. We further analyzed the guessability (a common strength metric) of selfreported and pen-and-paper as well as for different demographic groups and find that overall, the strength of patterns is very similar. Overall, these results suggest that while in-lab surveys for Android graphical passwords using pen-and-paper are a reasonable substitute for real/in-the-wild data, there are likely subtle ecological differences that need some accounting as well as consideration of the demographic sampling, particularly for gender.